1. Field of the Invention
The present invention relates to an image generating apparatus for generating computer graphics (hereinafter, referred to as "CG") and expanding a high efficient coded digital moving picture (hereinafter, referred to as "compressed moving picture expansion"). More specifically, the present invention relates to an image generating apparatus capable of both generating CG in real time by texture mapping and expanding a high efficiency coded digital moving picture by using inter-frame motion compensation, a moving picture expansion mapping apparatus using the image generating apparatus, and a multimedia apparatus.
2. Description of the Related Art
Recently, in order to generate CGs to be displayed on a CRT display and the like in home TV game equipment, personal computers and work stations, a method for mapping previously provided image data (hereinafter, referred to as a "texture image") to a polygonal region (hereinafter, referred to as a "polygon"), that is, a "texture mapping method" is employed. With the texture mapping method, even when a complex three-dimensional stereoscopic shape is approximately drawn with a small number of polygons, a relatively high quality image can be advantageously obtained.
On the other hand, in order to display moving picture data on a CRT display and the like in personal computers or audio-visual devices, a method for displaying previously compressed moving picture data in real time while displaying the moving picture data is desired. By using the compressed moving picture data, both the capacitance of a storage medium and the amount of transmitted data can be reduced.
In order to effectively compress the moving picture data, a method is used for coding only the differential data of the image data over a plurality of frames, that is, a so-called inter-frame prediction coding is employed. The inter-frame prediction coding includes "forward prediction coding", "backward prediction coding" and "bidirectional prediction coding". These methods use respectively different combinations of images to obtain the differential data.
In a sequence of input images, an image to be compressed is referred to as an "input image", an image input prior to the input image is referred to as a "past image", and an image input after the input image is referred to as a "future image". In the forward prediction coding, the differential data between the input image and the past image is coded. In the backward prediction coding, the differential data between the input image and the future image is coded. In the bidirectional prediction coding, the differential data is coded by using the past image and the future image. Hereinafter, the past image and the future image are collectively referred to as a reference image.
Furthermore, a motion compression processing procedure is performed in order to increase compressibility. In the motion compression processing procedure, a difference at the same position in the image is calculated in order to obtain the differential data. Simultaneously, an input image is divided into image blocks having the same size (for example, 16.times.16 pixels), and the position at which the difference becomes minimum is obtained for each block while horizontally and vertically shifting the position within a corresponding reference image. Then, the block position, at which the minimum difference is established, and the differential data are coded. In this method, in order to obtain the maximum compressibility, the coding is performed while suitably selecting any one of a "forward prediction", "backward prediction" and "bidirectional prediction", or "no prediction" type of motion prediction code. In some cases, the position of the reference image block is obtained not only with one pixel precision, but also with half-pixel precision.
Recently, in the texture mapping method, a method using a compression coded moving picture as a previously prepared texture image, that is, a method for mapping an expanded moving picture on a polygon has been employed. An example of a conventional image generating apparatus for realizing this texture mapping method is disclosed in Japanese Laid-Open Patent Publication No. 6-162165.
The method disclosed in the above patent publication provides an image processing procedure method in which a moving picture can be used as a texture image instead of a conventionally used static image such as a geometrical pattern, a photograph or a handwritten drawing. As a result, CG images of great variety now can be generated. The conventional image generating apparatus includes: data reproduction means for reproducing the compressed moving picture data from an information storage medium; image data expansion means for expanding the compressed moving picture data reproduced from the data reproduction means; and image synthesis means for receiving the expanded image data from the image data expansion means via a bus line and for synthesizing the image data.
With the configuration of the aforementioned conventional example, however, the following problems arise.
First, the configuration of a conventional image generating apparatus will be described with reference to FIG. 27. In FIG. 27, a system bus 2401 serves as a medium for the transmission of data between blocks. A CPU 2402 controls the entire system. A main memory 2403 stores programs and data. An image expansion circuit 2404 decompresses the compressed moving picture data. A CD-ROM decoder 2405 reproduces the data from a CD-ROM 2409 serving as an information storage medium for the image data for the image data or the like. An image synthesis circuit 2406 synthesizes the expanded moving picture data by a texture mapping processing procedure and stores the data to a frame memory 2410. A coordinate transform circuit 2407 obtains coordinates for image synthesis. A DMA controller 2408 transforms the data on the system bus. A frame memory 2410 provides a work memory for image synthesis. A DA converter 2412 performs DA conversion on a signal for displaying the result of image synthesis.
Next, the operation of the image generating apparatus will be described. The compressed moving picture data stored in the CD-ROM 2409 is read out by the CD-ROM decoder 2405 and reproduced. The reproduced compressed moving picture data is transferred to the image expansion circuit 2404 via the system bus 2401. The image expansion circuit 2404 decompresses the compressed moving picture data. The expanded moving picture data is transmitted to the image synthesis circuit 2406 via the system bus 2401. At the same time, coordinates of a plotted image are obtained through the CPU 2402 and the coordinate transform circuit 2407, and are specified to the image synthesis circuit 2406 via the system bus 2401. The image synthesis circuit 2406 synthesizes the moving picture data through texture mapping and the synthesized data is transmitted from the image expansion circuit 2404 onto the coordinates on the frame memory 2410 as directed by the CPU 2402. The synthesized image in the frame memory 2410 is read out by the image synthesis circuit 2406, and is then transmitted to the DA converter 2412. The digital signal is converted by the DA converter 2412 to an analog signal, and an analog signal is output.
FIG. 40 shows an inner configuration of the image expansion circuit 2404 shown in FIG. 27. Hereinafter, the operation of motion compensation processing procedure will be described with reference to FIG. 40.
The compressed moving picture data read out from a CD-ROM decoder is input to the image expansion circuit 2404 via the system bus 2401, as a unit of an image block consisting of 16.times.16 pixels, for example. The compressed image block data input to the image expansion circuit 2404 through a local bus 3014 is first transformed to an orthogonal transform coefficient image 3002 by a variable-length inverse coding circuit 3001. Subsequently, the orthogonal transform coefficient image 3002 is transformed into a differential image block by an orthogonal transform circuit 3003, said orthogonal transform coefficient image 3002 is in turn stored in a differential image memory 3004. The compressed image block data to be input to the image expansion circuit 2404 includes not only a differential image component but also motion compensation data required for decoding the image block. The motion compensation data corresponds to data indicating the type of motion prediction (i.e., any one of "forward prediction", "backward prediction", "bidirectional prediction" and "no prediction coding") used to code the image block, and the data indicating the type of the position of a reference image block used for the prediction. Based on the contents of the motion compensation data, the motion compensation processing procedure is subsequently performed in a motion compensation circuit 3013. The image block at the position corresponding to the data indicating the position of the reference image block is read out from a reference image memory 3012 via image reading circuits 3007 and 3008. For example, in the case where the image block to be transformed is a block coded by forward prediction, an image block read out from a past image memory 3005 is selected by an average/selection circuit 3009, and is input to an addition circuit 3010. The addition circuit 3010 adds the past image block data and the differential image block data, and the result is written to a generated image memory 3011. In the case where the image block to be transformed is an image block coded by backward prediction, an image block read out from a future image memory 3006 is selected and added to a differential image block. In the case where the image block to be transformed is an image block coded by bidirectional prediction, the average/selection circuit 3009 takes an average of the image block read out from the past image memory 3005 and the image block read out from the future image memory 3006 for each pixel. The result is transmitted to the addition circuit 3010 which adds the result to a differential image block. The result of the addition is stored in the generated image memory 3011. In the case where the data indicating the position of the reference image block is given with half-pixel precision, the image block reading circuits 3007 and 3008 have a reading function corresponding to half-pixel precision, for example, a function of reading out an image block while calculating an average of adjacent two pixels. The expanded moving picture data stored in the generated image memory 3011 is transmitted to the image synthesis circuit 2406 via the system bus 2401. The contents of the generated image memory 3011 are also transmitted to the past image memory 3005 or the future image memory 3006 in the reference image memory 3012 so as to be used for subsequent moving picture expansion. The inner configuration of the conventional image expansion circuit and the operation of motion compensation processing procedure are as described above.
Next, the operation of conventional texture mapping processing procedure performed in the image synthesis circuit 2406 and the frame memory 2410 shown in FIG. 27 will be described with reference to FIG. 41.
The moving picture data transmitted to the image synthesis circuit 2406 is temporarily stored in the frame memory 2410 as texture image data 3101 in the case where the image synthesis circuit 2406 performs the texture mapping operation. Then, the texture image 3101 is read out and transformed by the image synthesis circuit 2406, and is stored (mapped) in the region of a generated image 3102. One pixel 3104 in the texture image 3101 is input to a first multiplication circuit 3107 in the texture mapping circuit 3106. Simultaneously, a pixel 3105 in the generated image region 3102, to which the pixel 3104 is mapped, is input to a second multiplication circuit 3108. The first multiplication circuit 3107 multiplies the pixel 3104 by a multiplier .alpha. (where .alpha. is a numerical value equal to or larger than 0 and equal to or smaller than 1), while the second multiplication circuit 3108 multiplies the pixel 3105 by a multiplier (1-.alpha.). The results of these multiplications are input to an addition circuit 3109, which adds the input values. The result of the addition is stored at the position of the pixel 3105 in the generated image 3102. By sequentially performing the aforementioned operation for all of the pixels in the texture image, the texture mapping processing procedure of an original image in the generated image (which is generally a background image) and a texture image along with semi-transparent synthesis can be realized. The operation of texture mapping processing procedure performed in a conventional image synthesis circuit is as described above.
In the image generating apparatus having the configuration as described above, the compressed moving picture data is transmitted to the image expansion circuit 2404 via the system bus 2401 so as to decompress the moving picture. Then, the expanded moving picture data is transmitted to the image synthesis circuit 2406 via the system bus 2401. The amount of expanded moving picture data is generally extremely large (in the case of a moving picture including 320.times.240 pixels, about 7 MB/second), and therefore applies a large load on the data transmission ability of the system bus 2401. In addition, in the case where the moving picture is compressed with high efficiency, the inter-frame motion prediction technique is often used. In order to decompress the moving picture compressed based on this technique, it is necessary for the image generating apparatus to include the frame memory. More specifically, if the image expansion circuit 2404 and the image synthesis circuit 2406 are separately provided, a frame memory for image expansion processing procedure and the frame memory 2410 for plotting the image should be also separately provided. Moreover, since the image expansion circuit 2404 and the image synthesis circuit 2406 are connected to each other via the system bus 2401, control such as timing control of the data transmission is complicated. As a result, the system configuration will be complicated.
As described above, the conventional image generator has a number of problems.